Skip to content

Macos playback issue #1483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
richiemcilroy merged 9 commits into from
Jan 4, 2026
Merged

Macos playback issue #1483

richiemcilroy merged 9 commits into from
Jan 4, 2026

Conversation

@richiemcilroy
Copy link
Member

@richiemcilroy richiemcilroy commented Jan 4, 2026 •
edited by greptile-apps bot
Loading

Always use main thread WebGPU for canvas initialization to fix macOS playback and ensure consistent NV12 frame handling.

macOS playback was broken because the canvas was transferred to a worker, and WebGPU in the worker context was failing to render NV12 frames correctly after recent Windows optimizations. By forcing all platforms to use the initDirectCanvas path, we leverage the existing main thread WebGPU renderer which correctly handles NV12 frames and has a robust canvas2d fallback. The frame-worker.ts change adds a specific canvas2d fallback for queued WebGPU frames in the worker if WebGPU isn't initialized, ensuring robustness.

Open in Cursor Open in Web

Greptile Summary

  • Fixes macOS video playback by standardizing all platforms to use main thread WebGPU canvas initialization instead of worker-based rendering that was failing on macOS
  • Adds canvas2d fallback mechanism in frame-worker.ts to handle WebGPU frames when WebGPU renderer is unavailable, including NV12 to RGBA conversion and stride correction
  • Optimizes video decoder performance by reducing buffer sizes, cache limits, and timeouts across multiple components to improve playback responsiveness and reduce memory usage

Important Files Changed

Filename Overview
apps/desktop/src/routes/editor/Player.tsx Removes platform-specific canvas initialization logic, forcing all platforms to use direct canvas path to fix macOS WebGPU worker context failures
apps/desktop/src/utils/frame-worker.ts Adds comprehensive canvas2d fallback for WebGPU frames with NV12 conversion and stride correction when WebGPU renderer unavailable
apps/desktop/src-tauri/src/frame_ws.rs Reduces frame downscaling aggressiveness from 50% to 75% to maintain better frame quality for NV12 handling

Confidence score: 4/5

  • This PR addresses a clear macOS-specific playback failure by consolidating rendering architecture and adding proper fallbacks
  • Score reduced due to performance implications from buffer size reductions across multiple components that could affect playback smoothness under load
  • Pay close attention to the decoder configuration changes in crates/rendering/src/decoder/mod.rs and crates/editor/src/playback.rs as the reduced timeouts and buffer sizes may impact performance

Sequence Diagram

sequenceDiagram
    participant User
    participant Player as Player.tsx
    participant EditorInstance
    participant Playback
    participant SegmentMedia
    participant Decoder as VideoDecoder
    participant Renderer
    participant WebSocketServer as frame_ws.rs
    participant FrameWorker as frame-worker.ts

    User->>Player: "Click play button"
    Player->>EditorInstance: "startPlayback(fps, resolution)"
    
    EditorInstance->>Playback: "start(fps, resolution)"
    Playback->>Playback: "Initialize frame cache and prefetch buffer"
    
    loop "Video playback loop"
        Playback->>SegmentMedia: "get_frames(segment_time, !hide_camera, clip_offsets)"
        SegmentMedia->>Decoder: "get_frame(time)"
        Decoder->>Decoder: "Decode video frame from file"
        Decoder-->>SegmentMedia: "DecodedFrame"
        SegmentMedia-->>Playback: "DecodedSegmentFrames"
        
        Playback->>Renderer: "render_frame(segment_frames, uniforms, cursor)"
        Renderer->>Renderer: "Apply effects, camera overlay, cursor"
        Renderer-->>WebSocketServer: "RenderedFrame (via frame_cb)"
        
        WebSocketServer->>WebSocketServer: "Convert to NV12, downscale, pack metadata"
        WebSocketServer->>FrameWorker: "Send binary frame via WebSocket"
        
        FrameWorker->>FrameWorker: "Parse frame, queue for rendering"
        FrameWorker->>FrameWorker: "Render to canvas (WebGPU or Canvas2D)"
        FrameWorker-->>Player: "Display frame"
        
        Playback->>EditorInstance: "PlaybackEvent::Frame(frame_number)"
        EditorInstance-->>Player: "Update playhead position"
    end
    
    User->>Player: "Click pause/stop"
    Player->>EditorInstance: "stopPlayback()"
    EditorInstance->>Playback: "stop()"
    Playback->>Renderer: "Stop message"
Loading

Context used:

  • Context from dashboard - CLAUDE.md (source)
  • Context from dashboard - AGENTS.md (source)

@cursor
Copy link

cursor bot commented Jan 4, 2026

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
Learn more about Cursor Agents

@richiemcilroy richiemcilroy marked this pull request as ready for review January 4, 2026 17:33
@richiemcilroy richiemcilroy merged commit a05d511 into main Jan 4, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@richiemcilroy @cursoragent